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ABSTRACT 

This is a report of a five-day training session on 
systems research for counselors, counselor educators, and 
supervisors. The training session was an advanced program dealing 
with the use of systems research for planning and evaluating 
counseling, counselor education, supervision and related programs. 
The primary aims of the program were to: (1) develop participants' 

knowledge and understanding of systems research concepts and 
principles; and (2) to develop participants' proficiency in using 
systems techniques for planning and evaluating counseling, counselor 
education, supervision, and related areas. Forty trainees 
participated in the program, which included didactic instruction and 
supervised practice. Evaluation revealed attainment of program 
objectives at a near criterion level for most participants. Program 
evaluation suggests that organization and management were generally 
satisfactory. (Author) 
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I. Introduction 

A. Problem 

This is a report of a five-day research training session on systems 
research, held from March 17 to 21, 1970 in New Orleans, Louisiana, immedi- 
ately preceding the annual meeting of American Personnel and Guidance 
Association. The presession was one of four research training sessions 
sponsored by American Personnel and Guidance Association in 1970 with sup- 
port from the U. S. Office of Education. The presessions were intended to 
meet the needs of individuals whose full-time employment in professional 
roles made long-term training difficult and impractical. The presession in 
systems research was designed as an advanced program for teaching the use 
of systems research in planning and evaluating, counseling, counselor edu- 
cation, supervision and related programs. 

B. Statement of need 

The forces of change and pressures of a changing society have created 
a critical need for improvement and innovation in guidance and counseling 
(Riccio and Walt, 1967; Wrenn, 1962, 1965; Wolfbein, 1967). The standards 
for counselors and counselor educators (Association for Counselor Education 
and Supervision, 1965; American School Counselor Association, 1965) call for 
professionalization with concomitant improvement and innovation in imple- 
menting the roles of counselor, counselor educator, and supervisor. The 
public demand for accountability forces the profession to look for scientific 
approaches to planning and evaluating counseling and guidance. The standards 
and the public demand for accountability are mandates for improvement-and 
reforms in counseling and counselor education (Loughary, 1965). Nearly a 
decade ago Wrenn (1962) challenged the profession to make a searching analy- 
sis of goals and to seek for new and effective ways to meet these goals. 

These two needs are as real and demanding in 1970 as they were in 1962. The 
presession in systems research was addressed to meeting these needs by 
equipping counselors, counselor educators, and supervisors with practical 
skills and theoretical knowledge for implementing systems research in local 
district, state department, or university settings. 

C. Rationale 

It was assumed that counseling and counselor education constitute 
systems, and that improvement in the systems could be achieved through appli- 
cation of the principles and techniques of systems research. It was assumed 
that judicious use of analysis, synthesis, simulation, and modelling could 
result in innovations and improvements in the counseling and guidance sys- 
tems which would meet the needs of the society and answer the mandate for 
accountability. It was assumed that an intensive training program dealing 
with conceptualization of systems, and application of systems research tech- 
niques and skills could train selected individuals to a sufficiently high 
level of performance so they could employ systems research in their profes- 
sional roles. 
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D. Purposes and Objectives 

The ultimate purpose of the presession in systems research was to 
improve counseling, counselor education, supervision and related areas 
through research . The immediate program purpose was to train selected 
participants in use of systems research principles and techniques for plan- 
ing and evaluating counseling, counselor education, supervision and related 
areas. The program purpose was implemented in two primary aims: 

(1) to develop participants' knowledge and understanding of systems 
research concepts and principles as revealed by meaningful definition of 
analysis, synthesis, simulation, modelling, feedback, and behavioral objec- 
tives . 



(2) to develop participants' proficiency in using systems tech- 
niques for planning and evaluating counseling and counselor education, as 
revealed by design of a closed loop system with element identification, 
feedback, logical sequence, and part-whole relationships and correct use of 
signal paths, arrowhead formation, rectangular blocks, descriptors, point 
numeric codes. F, FF, A, and error signals. 
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II. Method 

A. Design 

The program was designed to provide a carefully sequenced series of 
exercises and problems integrated with didactic instruction. It was assumed 
that factors influencing extent to which program aims could be achieved 
included participant background, staff competency, quality and quantity of 
information presented and practice provided. 

Participants were selected who met criteria for education and 
experience deemed essential for progress in the course. In staff selection 
the intent was to combine competencies of different instructors into a strong 
instructional team capable of didactic presentation and supervision over 
problem-solving activities. 

The amount and kind of information presented was controlled through 
the planned reference list and directed reading, including pre-conference 
preparation. 

B. Setting 

The training session was of five days duration, from March 17 to 21, 
1970, with daily programs from 8:00 A.M. to 4:30 P.M. Instructors were 
available from 7:00 to 10:00 P.M. for individualized assistance to partici- 
pants. 



C. Participants 

Forty-five participants were selected from sixty-three applicants 
for the presession. A<t£riti'dn of five left forty in attendance during the 
training program. Participant roster is given In Appendix A. 

Method of Selection 

Applications for the presession on systems research were evaluated 
against the following criteria: 

\ 

1. Employment as counseling specialists, counselor educators, 
supervisors, educational psychologists, researchers with responsibilities for 
research in counseling, counselor education, supervision, or related areas. 

2. Education and experience to benefit from the instructional 

program 



In selecting participants there was no discrimination on account of 
race, color, sex, or national origin^ 

Letters of invitation to participate in the presession on systems 
research were mailed by the directors to a select group of potential candi- 
dates. Those accepting the invitation received advance enrollment. A 
notice of the four presessions and one postsession sponsored by American 
Personnel and Guidance Association was carried in the March issue of 
Guidepost and direct mailing of application forms and information circulars 
was made to APGA members. 
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Ntoice was sent by the presession directly to applicants informing 
them of the status of their application. Applicants accepted for the program 
were required to file an Enrollment Form. 

Participant Characteristics 

There were males and females representing colleges and universities, 
local school districts, state departments of education, and agencies. 

Distribution of participants by sex, place of residence, educational 
attainment, place and nature of employment is given in Appendix B. 

C. Presession Staff 

The staff consisted of the director who implemented administrative 
and instructional responsibilities, five instructors and one assistant. 

Staff directory is given in Appendix C. 

D. Training Program 

The training program designed to achieve presession objectives was 
five days in duration, with daily sessions from 8:00 A.M. to 12:00 Noon, and 
1:00 P.M. to 4:30 P.M. Evening sessions were not scheduled as a required 
part of the program. The training facilities were open from 7:00 to 10:00 
each evening, with instructors on hand to work with individual trainees or 
groups of participants. A demonstration of computer simulation and presen- 
tation of mathematical modelling were given during the evening hours. 

Prior to the start of the presession, enrollees were sent a materi- 
als packet, with directions for pre-conference reading, seven reference 
materials, syllabus, and 'staff directory. Reference list is given in 
Appendix D, and syllabus is shown in Appendix E. 

The presession opened with an orientation to the training program. 
This was followed immediately by a pretest to determine extent to which 
participants already were capable of demonstrating the terminal behaviors 
defined in the program objectives. There were seven major elements in the 
program, following completion of the pretest: (1) instruction in basic 

concepts and principles to reinforce required pre-conference reading; 

(2) basic instruction in skill development; (3) advanced instruction in 
concepts and techniques; (4) practice in applying systems skills and tech- 
niques; (5) posttest to determine extent to where participants had pro- 
gressed toward criterion performance; (6) application of concepts and 
techniques in developing a solution to a real-life problem; and (7) presen- 
tation of models demonstrating systems research in counseling and counselor 
education. 



The learning activities implemented to achieve Aim 1, developing 
participants' understanding of systems concepts and principles included 
assigned reading, lecturer, slide-tape presentations, films, individualized 
activities with programmed material, and supervised practice on workbook 
exercises . 
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Activities to achieve Aim 2, developing participants' proficiency 
in using systems techniques and skills included film-tape presentation, 
and supervised practice on individual and group problems, including work 
on analysis, synthesis, and flow chart modelling. Advanced exercises and 
extra assignments were utilized to help meet individual needs. 

The daily schedule is shown in Appendix F. 
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III. Results 

A. Evaluation of Participant Performance 

Two measures were taken to evaluate participant performance against 
program objectives: an objective pre- poettest and subjective participant 

self-evaluation. Evaluation of the research training session was accom- 
plished by comparing pre- and posttest group profiles and by comparing pre- 
and posttest scores against criterion standards for acceptable performance. 
The pre- posttest was designed to sample behaviors defined by Aims 1 and 2, 
developing understanding of concepts and principles of systems research and 
developing proficiency in using systems techniques. The pre- posttest 
instrument consisted of three subtests, two of which sampled behaviors of 
concept understanding, and one sampled proficiency in using systems tech' 
niques . 



Table 1, Appendix G, shows the group profile for median scores on 
the pre- and posttest by program objectives. Inspection of Table 1 reveals 
that the posttest median scores for understanding of concepts exclusive of 
behavioral objectives were four and a half times larger than the pre-test 
median score. The median score for the understanding of behavioral objec- 
tives was double the pretest score. The posttest score on the test of skill 
proficiency was three times the pretest score. When the pre- and posttest 
scoreo were compared, against criterion standards for acceptable performance 
(Table 2, Appendix G), it was found that over three-fourths of the partici- 
pants developed understanding of concepts excluding behavioral goals at 
criterion level on the posttest, with ninety percent reaching criterion level 
on the posttest for skill performance. 

Self-evaluation against Aims 1 and 2 were taken by eliciting from 
participants responses to indicate their feelings about their progress to 
the training objectives. Table 3, Appendix G, reports results of participants' 
self-evaluations. Inspection of Table 3 reveals that 100 percent of partici- 
pants felt they had developed a significant understanding of systems concepts, 
with 97 percent reporting they felt they had acquired proficiency in using 
systems techniques. 

B. Evaluation of Program Organization and Administration 

A program evaluation was made to assess program management, by 
gathering data on learning activities, instructional materials, program con- 
tent, and program organization. 

Participants rated learning activities on a four-point scale, indi- 
cating degree to which the activity contributed to achievement of program 
goals. Mean ratings are reported in Appendix H, Table 4. Examination of 
data reported in Table 4 reveals all of activities were rated above the chance 
mean. The learning activities rated as most worthwhile in helping partici- 
pants reach program goals were problem solving tasks, assigned readings, and 
conferences with staff. 
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Evaluation of instructional materials was made by participant 
rating on a four-point scale of six references which were required reading 
for the course. Mean ratings are reported in Appendix H, Table 5. Inspec- 
tion of Table 5 reveals that all references weee rated above the chance 
mean. The references rated as most valuable were Systems techniques for 
programs of counseling and counselor education b y T. A. Ryan, and Preparing 
instructional objectives by Mager, with the next highest rated references 
being LOGOS: A system language for flowchart modeling by Silvern.. 

Program content was evaluated by participant rating on a four- 
point scale of each program unit in terms of contribution to program goals. 
Mean ratings are reported in Appendix H, Table 6. Inspection of Table 6 
reveals that units considered most valuable were conceptualization of system 
in model form and conceptual analysis and synthesis. All units were rated 
above the chance mean. 

Program management was evaluated by participant rating of aspects 
of program organization and management, including program information, meals 
and lodging staff qualifications, time utilization, climate for learning, 
and physical facilities. Participant ratings of program management are 
reported in Appendix R, Table 7. Results in Table 7 indicate same dissatis- 
faction with the program information, meals and living arrangements, time 
allocation, and physical arrangements. Ihere was 100% satisfaction with the 
program as a whole, as indicated by response to the questions concerning 
scheduling of a similar presession next year. The scope and sequence of 
learning and program expectations. 
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IV. Discussion 

A. Purpose 

The primary purpose of this American Personnel and Guidance 
Association research training session on systems research was to equip 
counselors, counselor educators, and supervisors with practical research 
skills and theoretical knowledge needed to achieve improvement and innova- 
tion in counseling, counselor education and related areas* The program 
purpose was implemented in two primary aims: (1) developing participants' 

understanding of systems concepts and principles; and (2) developing 
participants' proficiency in using systems techniques. 

B. Results 

Analysis of results from criterion tests indicates that the aim 
of developing participants' knowledge and understanding of systems concepts 
and principles was achieved by over three-fourths of the participants, with 
ninety percent of participants reaching criterion level in use of systems 
techniques. Analysis of test data revealed that twenty percent of partici- 
pants started at criterion level on understanding of behavioral objectives. 
At the end of the training program, fifty percent had reached performance 
standards on defining behavioral objectives. The data reveal that none of 
the participants was at criterion level on understanding of other systems 
concepts at the start of the program. Seventy-eight percent reached crite- 
rion level at the end of the presession. The difference between the 
development of participants' understanding of behavioral objectives and 
their understanding of other systems concepts is explained by the nature of 
the program. The program design assumed a prior understanding of behav- 
ioral objectives^' and the learning activities were not planned to 
emphasize understanding of behavioral objectives. The indication that 
twenty percent of the participants had an understanding of behavioral objec- 
tives of the beginning of the program indicates the extent to which pre- 
conference reading on behavioral goals was effective. A parallel can be 
shown with developing participant skill proficiency in using systems tech- 
niques. On the pretest seventeen percent of participants met criterion 



<i>The program of instruction on the presession assumes a prior under- 
standing of certain basic concept and principles, and ability to perform 
certain activities with ease and ccmpetence. In order to derive maximum 
benefit from the training program, participants must have a thorough under- 
standing of the language of systems research, and must be able to opera- 
tionalize mission goals and to define behavioral objectives . It is assumed 
that before the presession begins participants will be capable of defining 
problems, stating objectives in behavioral terms, and identifying alterna- 
tives to implement the objectives. The references listed are intended to 
provide a means by which participants can acquire the prerequisite knowledge 
and skills which are assumed for this program. Reference annotations are 
provided to assist in directing reading activities so that optimum use can 
be made of participants' reading time prior to the start of the presession. 
Quoted from page 1, Selected References, APGA-008, February 18, 1970. 
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levels of use of systems techniques. This suggests that one-fifth of 
participants managed to learn from the basic system skills from the pre- 
conference preparation. However, in looking at the participant, understand- 
ing of systems concepts, with none of the participants meeting criterion 
levels on the pretest, it seems that the learning of systems skills through 
pre-conference directed reading failed to develop an understanding of 
systems principles and concepts. The continued independent use of systems 
research for improvement and innovation of counseling, counselor education 
and related areas requires more than rote skill. There must be understand- 
ing of assumptions underlying use of the systems techniques, to permit the 
researcher to know how and when to use systems techniques and to be able to 
interpret results of systems analysis and synthesis. Results of the pre- 
posttests suggest that an important outcome of the training session was 
the development of participant understanding of concepts and principles of 
systems research, along with developing proficiency in using systems tech- 
niques. 



The evaluation of program management reflects that organization and 
administration were generally satisfactory, despite difficulties encountered 
as a result of the very late funding of the proposal and concomitant delays 
in announcing the program. The dissatisfaction with the time available for 
the presession can be taken as a positive endorsement of the program, rather 
than a criticism. The essence of the responses indicating dissatisfaction 
with amount of time for the presession has the effect of saying more time 
is desired to devote to the topic. 

The dissatisfaction with pjysical facilities reflects in part the 
problems encountered, not only in the hotel facilities but also in the mov- 
ing from one location to another in the course of the program. 

Participant evaluation of instructional materials, techniques, and 
staff reveal general satisfaction with these components of the program. 

There was overwhelming endorsement of the program, as indicated by one 
hundred percent response to the question, "Did the program meet your expec- 
tations?" 



The data from participant and program evaluation suggest that the 
research training programs conducted in cooperation with the annual meeting 
of American Personnel and Guidance Association, meets a very real need. 

The comments of participants (Appendix I) clearly indicate that the topic of 
systems research is one of interest and value to members of APGA. 
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UNIVERSITY OF HAWAII 

EDUCATION RESEARCH AND DEVELOPMENT CENTER 



Hon olulu, Hawaii 96322 
1776 University Avenue 



AMERICAN PERSONNEL AND GUIDANCE ASSOCIATION 1970 FRESESSION 
SYSTEMS RESEARCH FOR COUNSELORS, COUNSELOR EDUCATORS, AND SUPERVISORS 

March 17 to March 21, 1970 
Jung Hotel, New Orleans, Louisiana 



PARTICIPANT ROSTER 



1. Miss Etheline J. Acox 
Counselor 

Orleans Parish School Board 

703 Carondelet Street 

New Orleans, Louisiana 70126 

2. Dr. Theodore Andreychuk 
Chairman, Department of Psychology 
Texas Tech University 

Lubbock, Texas 79413 

3. Dr. James D. Beck 
Professor of Education 
Chairman, Department of Guidance 
Florida A & M University 

Box 321 (FAMU) 

Tallahassee, Florida 32307 

4. Dr. Keith Bell 

Dean of Students, Professor of 
Psychology 

Seattle Pacific College 
Seattle, Washington 98119 

5. Dr. George C. Beanie r 
Chairman, Division of Counselor 

Education 

North Texas State University 
Denton, Texas 76203 

6. Mr. Bruce W. Bergland 
Assistant Professor of Education 
Northwestern University 
Evanston, Illinois 60091 

7. Dr. Herman Boroughs 
Coordinator, Graduate Studies 
College of Education and Psychology 
University of Southern Mississippi 
South Station, Box 12 
Hattiesburg, Mississippi 39401 



8. Mrs. Emma J. Brown 
Counselor 

Employment Security Division- 
Monticello 
P. 0, Box 30 

Monticello, Arkansas 71655 

9. Dr. William Carse 

Assistant Professor of Educational 
Psychology 
University of Texas 
3413 Bridle Path 
Austin, Texas 78703 

10. Mrs. Beverly B. Clark 
Coordinator, Law Enforcement 

Programs & Curriculum Counselor 
University College, University of 
Maryland 

College Park, Maryland 20742 

11. Dr. David R. Cook 
Professor and Chairman 
Department of Counselor Education 
Northeastern University 

201 Cushing Hall 

Boston, Massachusetts 02115 

12. Dr. Matthew Cooper 

Professor of Psychology and Guidance 
Texas Southern University 
3201 Wheeler Avenue 
Houston, Texas 

13. Dr. Norman M. deMoose 
Director of Counseling 
Student Personnel Department 
Staten Island Community College 
715 Ocean Terrace 

Staten Island, New York 07657 
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14. Dr, Edward P. Dworkin 

Center for Vocational and Technical 
Education 

Ohio State University 
1900 Kenny Road 
Columbus, Ohio 43212 

15. Sheldon Eisenberg 
Assistant Professor-Education 
Syracuse University 

804 University Avenue -3rd Floor 
Syracuse, New York 13210 

16. Dr. James Engelkes * 

Department of Counseling, Personnel 

Services and Educational Psychology 
Michigan State University 
East Lansing, Michigan 48823 

17. Dr. William P. Ewens 
Professor of Education 
Oklahoma State University 
Stillwater, Oklahoma 74074 

18. Dr. Irvine C. Gordon 
Associate Professor o5 Guidance 

and Education 
Drawer Z 

Prairie View, Texas 77445 

19. Dr. Robert C, Gowdy 
Assistant Dean, Junior Division 
Louisiana State University 

Lake Front, New Orleans, La. 70122 

20. Dr, Richard V. Hall 
Assistant Professor 
Department of Education 
Midwestern University 
3400 Taft, Box 220 
Wichita Falls, Texas 76308 

21. Lt. Richard B. Haynes 
Director of Counseling 

U.S. Army Ordinance Center and 
Schoo 1 

S & F 319, Aberdeen Proving Ground 
Aberdeen, Maryland 21005 

22. Mr. N. Kenneth LaFleur 
Assistant Professor 
School of Education 
University of Virginia 
Peabody Hall 
Charlottesville, Va. 

* non-attendance 




23. Mrs. Virgil C. Kenney 

Director of Testing and Counseling 
Texas Southern University 
3201 Wheeler Avenue 
Houston, Texas 77004 

24. Dr. De Wayne J. Kurpius 
Counselor Educator 
Indiana University 
Bloomington, Indiana 47401 

25. Mr. Robert M. Lewis * 

Teaching Assistance and Practicum 
Supervisor 

University of Wisconsin 
1914 Rowley Avenue 
Madison, Wisconsin 53705 

26. Dr. Florence M. Lloyd * 

Chairman, Department of Education 
St, Francis College 

2701 Spring Street 
Fort Wayne, Indiana 46808 

27. Mrs. Laverne Marr 

Assistant to Director, Psychological 
Services 

Dallas Independent School District 
3700 Ross Avenue 
Dallas, Texas 75204 

28. Mr. Alan McNabb 
Graduate Student 
Indiana University 

School of Education, Room 105 
Bloomington, Indiana 47401 

29. Dr. Ralph A. Meyer ing *£ 

Professor of Education and Psychology 
Illinois State University 
Normal, Illinois 61761 

30. Dr. Earl J. Moore 
Associate Professor 
University of Missouri 
24 Hill Hall 

Columbia, Missouri 65201 

31. Dr. C. Eugene Morris 
Professor of Education 
Graduate School of Education 
Guidance and Counseling Department 
Long Island University 
Brookville, New York 11548 
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32. Dr. C. N. Palms 
Counselor Educator 
The Fort Valley State College 
Fort Valley, Georgia 31030 

33. Dr. L. E. Patmore * 

Professor of Education and Psychology 42. 
Department of Education 

Eastern Washington State College 
Cheney, Washington 99004 



34. Dr. Helen Prouty 
Professor of Education 
San Diego State College 
San Diego, California 92115 

35. Sister Agnes Lucile Raley 
Counselor Educator and Chairman 
Psychology Department 
Spalding College 

851 South 4th Street 
Louisville, Kentucky 40203 

36. Mrs. Janice P. Rivers 
Counselor 

Airport High School 
Eagle Drive West Cola 
South Carolina 29206 

37. Mrs. Mary E. Sand 
Academic Counselor 
Louisiana State University 

Lake front, New Orleans, La. 70122 

38. Dr. Otto Spielbichler 
Assistant Professor 
University of Maryland 
College Park, Maryland 20742 

39. Dr. Mary E. Stephenson 
Professor 

Psychology and Education 
Henderson State College 
Arkadelphia, Arkansas 71923 

40. Dr. Jack C. Sturges 
Assistant Professor 
Center Teacher Education 
Tulane University 

New Orleans, Louisiana 70118 
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41. Miss Nancy A. Tyler 
College Counselor 
Flint Community Junior College 
1401 East Court Street 
Flint, Michigan 48503 

Sister Catherine Walker 
Professor of Education * 

Our Lady of the Lake College 
411 S.W. 24th Street 
San Antonio, Texas 78207 

43. Mr, Willie L. Watson 
Counselor 

W. E. Parker High School 
Edgefield, South Carolina 

44. Dr. James M. Williams 
Assistant Professor 
Educational Psychology Department 
University of Alabama 
University, Alabama 35486 

45. Dr. Earl D, Wilson 
Assistant Professor 
University of Nebraska 
Lincoln, Nebraska 




\ 



* non-attendance 



APPENDIX B 



PARTICIPANT CHARACTERISTICS 



Sex Number 

Male 27 

Female 13. 

.1 / * 

N - 40 



Highest Educational Degree Attained 



Degree Number 



Bachelors degree 2 
Masters degree 10 
Doctoral candidate 3 
Doctoral degree 25 



N » 40 



Place of Residence 



Nature and Place of Employment 



Region State 



Number Employer Position Number 



South 

Florida 
Alabama 
Mississippi 
South Carolina 
Georgia 



Southwest 

Arkansas 

Louisiana 

Oklahoma 

Texas 



Bast 

New Jersey 
New York 
Maryland 
Kentucky 
Massachusetts 



Midwest 

Missouri 

Ohio 

Michigan 

Nebraska 

Indiana 

Illinois 




University of College 
Dean 

Assistant Dean 
Chairman 
Director 
Processor 

Associate Professor 
Assistant Professor 
Coordinator 
Instructor 
Research Associate 
Counselor 



Local School 

Counselor 

Assistant Director 



Agency* Military 

Counselor 

Director 



1 

1 

4 

1 

9 

4 

9 

1 

l 

1 



34 



2 

1 



3 



2 

1 



3 



1 

1 

2 

1 

2 

1 



8 



N * 40 



West 

Washington 1 

California _1 

2 
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APPENDIX C 



UNIVERSITY OF HAWAII Honolulu, Ha w aii 96822 

EDUCATION RESEARCH AND DEVELOPMENT CENTER 1776 University Avenue 

AMERICAN PERSONNEL AMD GUIDANCE ASSOCIATION 1970 PRESESSION 
SYSTEMS RESEARCH FOR COUNSELORS, COUNSELOR EDUCATORS, AMD SUPERVISORS 

March 1.7 to March 21, 1970 
Jung Hotel, New Orleans, Louisiana 



STAFF DIRECTORY 

Director 

T. A. Ryan, Researcher/professor, Education Research and Development 
Center, University of Hawaii, Honolulu, Hawaii 96822 



Instructors 

Donald G. Hays, Director, Pupil Personnel Services, Fullerton Union 

High School District, District Education Center, 211 West Commonwealth 
Avenue, Fullerton, California 92632 

Ray E. Hosford, Associate Professor of Education, University of California, 
Santa Barbara, California 93106 

James W, Lawrence, Assistant Researcher, Education Research and Development 
Center, University of Hawaii, Honolulu, Hawaii 96822 

Leonard C. Silvern, President, Education and Training Consultants, Co., 

815 Moraga Drive, Los Angeles, California 90049 

Norman R. Stewart, Associate Professor, College of Education, Department 
of Counseling, Personnel Services and Educational Psychology, 

448 Erickson Hall, Michigan State University, East Lansing, Michigan 
48823 

Bob B. Winborn, Associate Professor, College of Education, Department of 
Counseling, Personnel Services and Educational Psychology, 436 Erickson 
Hall, Michigan State University, East Lansing, Michigan 48823 
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APPENDIX D 



U NIVERSITY OF HAWAII Honolulu, Hawaii 96322 

EDUCATION RESEARCH AND DEVELOPMENT CENTER 1776 University Avenue 

AMERICAN PERSONNEL AND GUIDANCE ASSOCIATION 1970 PRESESSION 
SYSTEMS RESEARCH FOR COUNSELORS, COUNSELOR EDUCATORS, AND SUPERVISORS 

MAreh 17 to March 21, 1970 
Jung Hotel, New Orleans, Louisiana 



SELECTED REFERENCES* 



The program of instruction in the presession assumes a prior understanding of 
certain basic concept and principles, and ability to perform certain activities 
with ease and competence. In order to drive maximum benefit from the training 
program, participants must have a thorough understanding of the language of 
systems research, and must be able to operationalize mission goals and to define 
behavioral objectives . It is assumed that before the presession begins partici- 
pants will be capable of defining problems, stating objectives in behavioral 
terms, and identifying alternatives to implement the objectives. The references 
listed are intended to provide a means by which participants can acquire the 
prerequisite knowledge and skills which are assumed for this program. Reference 
annotations are provided to assist in directing reading activities so that optimum 
use can be made of participants* reading time prior to the start of the presession. 

ABanathy, B, Instructional systems. Palo Alto, California: Fearon, 1968. 

A good overview of systems approach. Easy reading. Should be studied by 
everyone to insure thorough under standing of the nature of systems research. 

The appendix is particularly good. 

Bogus law, R. The new Utopians; A study. of system design and social change. 

Englewood Cliffs, N.J. : Prentice-Hall, 1965. 

An overview of systems approach. Intermediate reading level. 

Buckley, W. (Ed.) Modern systems- research for behavioral scientists. Chicago: 

Aldine Publishing Co. , 1968. 

A collection of articles dealing with systems theory. Advanced reading. 

Carter, L. F. Systems approach to education: Mystique or reality. Educational 
Technology, 1969, 9, 22-31. 

Gives an overview of the systems approach, with discussion of pros and 
cons from using the systems techniques. 

^Churchman, C. T;\, The systems approach. New York: Delacorte Press, 1968. 

This brief discussion of systems approach gives an excellent overview of 
the total systems concept t which involves problem identification, objectives 
definition, alternatives, identification and evaluation. This reference 
helps to point up the way in which flowchart modeling and simulation techniques 
implement the systems concept. Should be studied by all participants. 



*Items marked with asterisks are included in participants' Materials Packet. 
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Cooley, W. W. and Hummel, R. C. Systems approaches in guidance.. Review of 
Educational Research , 1959, 39, 351-362. 

Relates systems techniques to guidance* Easy reading* 

Educational Technology , 1969, 9, No. 3, 1-77. 

This special issue of Educational Technology is devoted to counseling 
technology. 

Eraut, M. R. An instructional systems approach to course development. 

AV Communication Revi ew, 1967, 15, 92-101. 

Relates the techniques of systems research to course development. 

Gagne, R. M. Educational objectives and human performance. In Krumboltz, J. D. 
(Ed.), Learning and the educational process . Chicago: RAnd McNally, 1965. 

Pp. 1-24. 

Discusses definition of objectives. Easy reading. 

*Mager, R. F. Preparing instructional objectives . Palo Alto, California: Fearon, 
1962. 

This book tells how to prepare behavioral objectives. It is mandatory that 
each participant be able to define objectives in behavioral terms. The 
principles discussed in this reference must be thoroughly understood by each 
participant. Each participant must be able to demonstrate proficiency in 
defining objectives behaviorally. This can be accomplished by concentrated 
study of this reference, and practice in preparing behavioral objectives. 
Should be studied carefully by all participants. 

*Ryaii? T. A. Systems techniques for programs of counseling and counselor education. 
In Silvern, L. G. (Ed.), Applying systems engineering techniques to educa- 
tion and training. Educational Technology , 1969, 9, 1-17. 

This article describes the application of systems techniques in counseling 
and counselor education. It provides a frame of reference for the 
presession. Easy reading. Excellent bibliography on systems research'. 

Should be read by all participants. The other articles in this issue of 
Educational Technol ogy all relevant to the presession topic. Casual 
reading is recommended. 

*Silvern, L. C. System s engineering of education I: The evolution of systems 

thinking in education . Los Angeles: Education and Training Consultants, 

1968. 

This is the basic text for the course . Pages 111-129 should be studied 
carefully by all participants . The program assumes that participants will 
have read this material and have a thorough understanding of the concepts 
presented in these pages. 

♦Silvern, L. C. LOGOS: A system language for flowchart modeling. In Silvern, 

L. C. (Ed.), Applying systems engineering techniques to education and 
training. Educational Technology , 1969, 9, 13-23. 

Contains basic vocabulary for flowchart modeling. Should be studied by all 
participants. 



*Items marked with asterisks are included in participants' Materials Packet. 
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Thoresen, C. E. The systems approach and counselor education: Basic features 
and implications. Counselor Education and Supervision , 1969 
Discusses the application of systems techniques to counselor education. 

von Bertalanffy, L. Modern systems theory . New York: George Braziller, Inc. , 
1968. 

Deals with systems theory. Advanced reading. 

Wiener, N. Cybernetics . Cambridge, Mass.: Massachusetts Institute of Technology 
Press, 1961. 

Deals with cybernetics aspect of systems research. Advanced reading. 

Wiener, N. Human use of human beings . New York: Doubleday, 1954. 

Deals with cybernetics in relation to systems research. Advanced reading. 
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SYLLABUS 



I. Nature of Presession 

A* Description 

1. This presession in systems research is one of four research training 
sessions offered in 1970 by American Personnel and Guidance Association 
with support from the U. S. Office of Education. 

2. This training session is designed as an advanced program focusing on 
the use of systems research for planning and evaluating counseling, 
counselor education, supervision and related programs. 

3. The program has been planned to equip counseling specialists, counselor 
educators or supervisors, educational psychologists, and researchers 
performing substantive research in counseling, counselor education, 
supervision or related areas with practical skills and theoretical 
knowledge essential for implementing systems research at local district, 
state department or university levels. 

4. The course of study deals with conceptualization of systems research, 
application of systems research, techniques of systems research, and 
practical uses of systems research. 

B. . Staff 

1. Director; T. A. Ryan, University of Hawaii 

2. Instructors; Donald G. Hays, Fullerton Union High School District, 

Fullerton, California 

Ray E. Hosford, University of California, Santa Barbara 

James W. Lawrence, University of Hawaii 

Leonard C. Silvern, Education and Training Consultants, Co. 

Norman R. Stewart, Michigan State University 

Bob B. Winborn, Michigan State University 

C, Participants 

The session will be open to individuals in public schools, state depart- 
ments of education, and colleges and universities who satisfy the following 

criteria; 

1. Employment as counseling specialists, counselor educators or 
supervisors, educational psychologists, or researchers with 
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responsibilities for performing substantive research in counsel 
ing, counselor education, supervision or related areas. 

2. Education and experience background to give evidence of ability 
to profit from instructional program, 

D. Purpose and Objectives 

1. APGA rationale for conducting presessions is based on assump- 
tions that 

a. significant benefits accrue from improved and expanded 
educational research and 

b. short, intensive in-service training is effective for equip 
ping those in professional roles with advanced knowledge 
and specialized research skills. 

2. Purpose of the presession on systems research is to improve 
counseling, counselor education, supervision, and related areas 
through research. This program is designed to train selected 
participants in use of systems approach for planning and evalu- 
ating counseling, counselor education, supervision, and related 
areas, 

3. Primary aims of the presession are: 

a. to develop participants' knowledge and understanding of 
systems research concepts and principles. 

b, to develop participants' proficiency in using systems tech- 
niques for planning and evaluating counseling and counselor 
education. 

4. Objectives implementing the presession aims are: 

a. Given a multiple choice objective test, participants will 
demonstrate understanding of systems research concepts by 
being able to select from alternative endings the one 
ending which best completes the statement of definition 
or illustration of basic systems concepts such as system, 
analysis, synthesis, simulation, model, anasynthesis , flow- 
chart, synergism, logistics, and fidelity, with, an accept- 
able performance level set at 80% correct responses in a 
given time period. 

b. Given a multiple choice objective test, participants will 
demonstrate understanding of principles of systems research 
by selecting from among alternatives the one ending which 
best completes statements of principles or illustrates 
principles such as the rules for coding, lettering, and 
signal paths, with acceptable performance level set at 

80% correct responses in a given time period. 
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c. Given a narrative description of a problem situation, the 
participant will be able to convert this word description 
into a flowchart model with correct element identification, 
use of symbols, descriptors, signal paths, blocks, coding 
and lettering, with acceptable performance level in a given 
time limit set at 80% agreement with problem solution. 

d 0 Given a flowchart model of a problem situation, the parti- 
cipant will be able to convert this model into a narrative 
description, with acceptable performance level in a given 
time period set at 80% agreement with problem solution. 

e. Given criteria for defining behavioral objectives, and a 
set of objectives, participants will be able to determine 
which objectives are stated in behavioral terms and the 
extent to which criteria for defining objectives behaviorally 
are satisfied. 

II. Program Outline and Activities 

A. Outline , ^ 

1. Systems research 

a. concepts and principles 

b. definitions 

c. background 

2. Techniques of systems research 

a. analysis 

b. synthesis 

c. modeling 

3. Practice in using systems research 

a. general problems 

b. counselor education and counseling problems, simple and 
complex 

4. Application of systems research to real-life problem 
a. situations identified by participants 

B. Activities 

1. Program will be intensive and demanding, involving five full 
workdays in addition to independent study and informal group 
sessions during evening hours. 

2. Activities will include lecture, discussion, demonstration, and 
task groups. 

3. Supervised practice in use of systems research will occupy 
major part of program, with participants working individually 
and in groups on prepared problems. 
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III. Requirements 

A. Participation 

1, Participants will be required to attend and to participate in 
daily lecture-discussion periods. 

2. Participants will be required to participate in task groups. 

B. Reading 

1. Readi.tg requirements will be self-determined according to needs 
of individual participants. This training program assumes a 
starting background of prior knowledge and skill proficiency 
on the part of participants . The reading list has been pre- 
pared with this in mind and is intended to serve the purpose 

of directing participants to sources of information for use 
in overcoming specific knowledge or skill deficiencies. 

2. It is recommended that participants study the references in 
the Materials Packet. An individualized program of in-depth 
study should be undertaken by each participant according to 
individual needs for background knowledge and skill develop- 
ment, so all participants will be starting the program with 
prerequisite knowledge and skill capabilities needed to benefit 
from training. 

IV. Evaluation 

A. Participant evaluation will be based on 

1. pre- and posttest of use of basic principles of systems 
research covered in the training sessions; and 

2. self-evaluation by participants. 

B. Program evaluation will be made through participant opinions con- 
cerning materials, staff and organization. 
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COURSE OUTLINE 
(Tuesday, March 17, 1970) 



Morning 



8:00-8:10 


Opening 


To A, Ryan 


8:10-9:10 


Pre -Assessment 




9:10-9:30 


Introductions 




9:30-9:40 


Program Overview: Purposes, Objectives 

Procedures 


T. A, Ryan 


9:40-10:00 


Thf Systems Approach: Concepts and 

Principals 


T. A. Ryan 


10:00-10:20 


Break 




10:20-11:00 


Defining Goals and Objectives 


T. A. Ryan 


11:00-11:10 


Model for Producing a System 


Slide tape 


11:10-12:00 


LOGOS Language for Flowchart Modeling 


Slide tape and Indi- 
' vidualized Activity 


Afternoon 






1:00-1:50 


Systems Engineering of Learning 


Filmstrip tape 


1:50-2:00 


Question-and-Answer Period 




2:00-3:00 


Analysis as a Process 


Individualized 

Activity 


3:00-3:20 


Break 




3:20-4:05 


Systems Using Feedback 


Slide Tape 


4:05-4:20 


Discussion 




4:20-4:30 


Announcements and Assignments: problems 
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COURSE OUTLINE 



Morning 

8 : 00 - 8:10 

8:10-8:40 

8:40-9:10 

9:10-9:30 

9:30-10:00 

10 : 00 - 10:20 

10:20-10:50 

10:50-11:00 

11:00-11:30 

11: 30-1 2'.: 00 

Afternoon 

1:00-1:30 

1:30-3:00 

3:00-3:20 

3:20-4:20 

4:20-4:30 
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(Wednesday, March 18, 1970) 



Opening 

Evaluate Solutions to Problem 1 
Synthesis as a Process 
Systems including Synthesis and CAl 
Problem 2 

Break 

Evaluate Solutions to Problem 2 

Model for Producing a System Model 

Study Closed Loco Instructional Flowchart 
Modil: Boeing 

Problem 3 

Evaluate Solutions to Problem 3 

Problem 4. Satellite \ 

Break 

Evaluate Problems Solutions 
Announcements and Assignments Problem 5 



v a A. Ryan 
Staff 

Slide Tape 

L. C. Silvern 

Individualized 

Activity 

Staff 
Slide Tape 

Individualized 

Activity 

Staff 

Task Groups 
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COURSE OUTLINE 



Morning 

8:00-3:10 

8:10-9:00 

9:00-9:30 

9:30-10:00 

10:00-10:20 

10:20-11:00 

11 : 00 - 12:00 

Afternoon 

1:00-3:00 

3:00-3:20 

3:20-4:20 

4:20-4:30 



(Thursday, March 19, 1970) 



Opening 

Evaluate Problem Solutions-Problem 5 

Study Complex. Closed Loop Instructional 
Flowchart Model: Occupational Instruction 
and Government Based Information 

Problem 6 

Break 

Problem 6 

Evaluation of Problem Solutions 



T. A. Ryan 
Staff 

L. C. Silvern 
Task Groups 
Task Groups 



Problem 7 
Break 
Problem 7 

Announcements and Assignments 



Tusk Groups 

Task Groups 
T. A. Ryan 
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COURSE OUTLINE 



(Friday, March 20, 1970) 



Morning 

8 : 00 - 8:10 

8:10-10:00 

10 : 00 - 10:20 

10 : 20 - 11:20 

11 : 20 - 12:00 



Opening 

Evaluate Solutions to Problem 7 
Break 

Post -Assessment 
Real Life Problem 



Afternoon 

1:00-3:00 

3:00-3:20 

3:20-4:20 

4:20-4:30 



Real Life Problem 
Break . ' 

Real Life Problem 
Announcements and Assignments 
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T. A. Ryan 



Task Groups 

Task Groups 

Task Groups 
T. I, Ryan 



COURSE GUTLESS 



(Saturday, March 21, 1970) 



Morning 

8 : 00 - 8:10 

8 : 10 - 10:00 

10 : 00 - 10:20 

10:20-10:40 

10:40-12:00 



x Opening 

Evaluate Solutions to Real Life Problems 
Break 

Program Evaluation 
A Counseling Model 



Afternoon 

1 : 00 - 2-00 

2:00-3:00 



3:00 



Model for a District Testing Program 

Review and Preview: Systems Approach- 
Implications for Counseling, Guidance, 
Counselor Education and Supervision 

Adjournment 




- 5 - 



T. A. Ryan 



N, Ri Stewart 
B, B. Winborn 

D. G. Hays 
T. A. Ryan 

T. A. Ryan 
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EVALUATION OF PARTICIPANT PERFORMANCE 



Table I 

Comparison of Group Profiles of Median Scores 
for Pre- and Posttest by Program Objective 



Median Score 

Program Objective Pretest Posttest 



Developing understanding of systems concepts 



exclusive of behavioral objectives 


9.0 


40.0 


Developing understanding of behavioral objectives 


3.0 


7.5 


Developing proficiency in using systems techniques 


7.5 


22.0 


N 


* 33 


N = 28 



Table 2 

Performance Criterion Levels for Training Objectives 
and Percent of Participants Achieving Criterion Levels 





Criterion 


Levels of Acceptable 


Performance 


Objective 


Possible 

Score 


Criterion 

Level 


% Achieving 
Criterion Level 
Pretest Posttest 


Understanding concepts exclusive 
of behavioral objectives 


96 


36 


0 


78 


Understanding behavioral 
objectives 


17 


8 


20 


50 


Acquiring proficiency in using 
systems techniques 


40 


20 


17 


90 
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EVALUATION OF PARTICIPANT PERFORMANCE 



Table 3 

Participant Sglf Evaluation on Levels of 
Performance for Program Objectives 



Program Objectives 


Percent of Respondents 
Reaching Four Levels of Performance 


None 


Little Some 


Great 

Amount 


1. Amount of new knowledge about 
system research acquired 
during presession 


0 


0 72 


28 


2. Extent of proficiency in 
using systems techniques 
developed during presession 


0 


3 79 


18 
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PROGRAM EVALUATION 
Table 4 

Mean Rating of Training Program Learning Activities 



Learning Activity 


Mean Rating 
(Md = 3.39%) 


Individual staff conference 




3.69 


Individual problem 




3.55 


Assigned reading 




3.52 


Lectures 




3.42 


Audio-visual presentations 


\ 


3.37 


Programmed instruction 


\ 


3.27 


General discussion 




3.14 


Task group activities 


/ 


3.06 


Table 5 






Mean Ratings of Instructional Materials 




Instructional Material 


Mean Rating 
(Md = 3.45) 


Ryan, T. A. Systems techniques for 
programs of counseling and 
counselor education. 




3.69 


Mager, R. F. Preparing instructional 
objectives. 




3.68 


Silvern, L. C. LOGOS: A system language 
for flowchart modeling. 




3.52 


Silvern, L. C. Systems engineering of 
education I: The evolution of systems 
thinking in education. 




3.38 


Banathy, B. Instructional systems. 




3.30 


Churchman, C. W. The systems approach. 




2.96 



APPENDIX H 
PROGRAM EVALUATION 

Table 6 

Mean Ratings of Program Topics 



Mean Rating 



Program Topic (Md = 3,49%) 

Conceptualization of system in model form 3.72 
Conceptual analysis and synthesis 3.72 
Model for producing a systems model 3,62 
Problem: From Real Life Environment 3.61 
Systems using feedback 3.55 
Problem: Counselor Education 3.51 
Rules and symbols for flowchart modeling 3.48 
Illustrations of Systems Research 3.32 
Closed loop instructional system 3.26 
Problem: Guidance Management 3.07 
Problem: Satellite Communication 2.82 
Problem: LOGOS 2.77 
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PARTICIPANT EVALUATIONS OF PRESESSION 



This was a tremendous ly worthwhi le experience. I appreciated the invitation to 
participate and got even more out of the workshop than I had anticipated. 
Saturday's program (staff presentations) were worth the entire workshop. Be- 
sides being a delightful person, Dr. Ryan exemplified the most effective work- 
shop leadership I have ever experienced. Systematization par excellence! 



I would like to attend another session of Systems Research as I was informed 
before leaving the area in which I work, that Systems Research is being 
considered for counselors and in the training of Counselors State wide 1972. 

I feel that I could help in working with those who plan to do this and it will 
definitely increase my helping those clients we serve in the Rural Areas of 
Arkansas. 



This presession was most informative and helpful. My one regret about the 
whole is related to time, i.e., a longer period of involvement in this kind 
of activity would have been most beneficial to me. 



Possibly something at an advanced project level where we can be briefly updated 
and then work intensively on "real life" problems with the aid of consultants, 
we need a means of "inventorying" the developed materials in counseling, 
counselor education, and guidance that utilize systems (e.g. the M.S.U. Staff) 
so we don't need to work from scratch on everything. Perhaps an advanced 
presession could begin with a couple of days of presentations of these systems. 



It was an intensive, stimulating experience;, very exhausting, but worthwhile. 
I found it difficult to move from the simple introductory material to the 
advanced model Silvern presented. Additional slide-tape materials would be 
helpful. Appreciate the staff time and planning; the organization was paced 
so that the time did not drag. Successful workshop! 



I found the presession exciting and mentally stimulating, something missing 
in the military. The lack of acquired knowledge on my part is not due to the 
staff, but 1) a lack of prior study on my part and 2) a lack of prior knowledge 
on my part. Another session on systems research is needed, but before next 
year. A summer session is a must to check ra£ progress. I found the staff of 
the finest quality; youthful in thought and action. Hopefully I shall have the 
opportunity to work with them in my doctorate work. I suggest the Jung Hotel 
be dropped due to poor service and inadequate facilities (living). The most 
pleasant information I received was the fact that the staff will continue to 
be available via mail. I am indebted to APGA, Dr. Ryan and Staff for this 
Fresession. Let's pray the Army will let me impliment some of the ideas 
learned here! 
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